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We  have  recently  described  both  engineering  and  perceptual  techniques  for  assessing  the  temporal  resolution  (i.e. 
moving  image  blur)  of  display  systems  used  in  Air  Force  flight-simulator  applications.  These  techniques  have 
subsequently  been  used  on  a  variety  of  display  devices.  The  engineering  techniques  were  developed  primarily  to 
provide  an  objective  measurement  standard  analogous  to  those  accepted  by  the  displays  industry.  The  perceptual 
techniques  were  designed  to  extend  the  measurement  standard  by  assessing  the  response  of  human  observers  to 
relatively  simple  test  stimuli.  In  the  present  study,  we  have  attempted  to  validate  our  previous  techniques  by 
correlating  them  directly  with  performance  on  a  simplified  air-to-air  task  that  may  be  performed  in  high-fidelity  Air 
Force  flight  simulators.  This  flight  performance  data  was  obtained  by  asking  observers  to  judge  whether  or  not  an 
F-16  target  banked  in  a  direction  indicating  that  it  was  turning  toward  the  observer.  This  task  was  performed  for  a 
target  distance  of  1600  m,  three  target  speeds,  and  4  levels  of  display  temporal  resolution  (i.e.  4  levels  of  moving 
image  blur).  The  results  indicated  that  decreasing  temporal  resolution  led  to  poorer  performance  on  the  roll 
detection  task.  Flowever,  results  with  temporal  resolution  closer  to  that  of  a  CRT  resulted  in  significantly  improved 
performance.  Minimum  display  temporal  resolution  required  for  flight  simulation  visual  displays  is  discussed. 
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INTRODUCTION 

Digital  displays,  such  as  Liquid-crystal  displays 
(LCDs),  Liquid  Crystal  on  Silicon  (LCoS),  Digital 
Light  Projectors  (DLP),  FerroElectric  Liquid  Crystal 
on  Silicon  (FLCoS),  etc.,  have  relatively  high  spatial 
resolution,  but  their  temporal  resolution  is  limited 
compared  to  CRTs.  This  limited  temporal  resolution 
often  results  in  the  blurring  of  moving  images 
(Klompenhouwer  &  Velthoven,  2004;  Kurita,  2001; 
Nakamura  &  Sekiya,.  2001).  It  was  originally 
believed  that  moving-image  blur  was  due  to  the  long 
onset  and  offset  times  typical  of  LCDs,  which  were 
often  longer  than  the  frame  duration.  Flowever,  the 
onset  and  offset  times  of  LCDs  have  been  reduced 
significantly  over  the  past  ten  years,  and  so  their 
limited  temporal  response  seems  now  to  be  a 
consequence  of  the  sample-and-hold  property  related 
to  both  the  design  of  the  LCD  driver  circuitry  and  the 
LCD  itself  (Yamamoto,  Aono,  &  Tsumura,  2000). 
Further,  DLP  and  FLCoS  displays  have  very  fast 
response  times  (in  terms  of  onset  and  offset),  but  still 
exhibit  significant  moving  image  blur.  The  severity 
of  the  moving  image  blur  on  displays  of  these  types 
is  such  that  they  are  unsuitable  for  flight  simulation 
and  training  applications. 

In  previous  work  (Winterbottom,  et  al,  2004,  2006, 
2007)  we  described  a  relatively  simple  procedure  for 
measuring  display  spatial  and  temporal  resolution 
and  observed  moving  image  blur.  Although  other 
procedures,  such  as  the  Motion  Picture  Response 
Time  (MPRT),  have  been  proposed  (see  e.g.  Igarashi, 
et  al,  2003;  Oka,  &  Enami,  2004;  Igarashi,  et  al, 
2004),  they  are  generally  more  complex,  require 
more  expensive  equipment,  and  do  not  directly 
measure  perceived  blur.  We  also  reported  that  LCD 
and  LCoS  projectors  equipped  with  mechanical 
shutters  that  reduced  pixel  hold-time  resulted  in  a 
significant  reduction  in  perceived  blur.  Further, 
experienced  pilots  who  viewed  the  resulting  imagery 
rated  moving  image  quality  as  significantly  higher 
when  the  shutter  was  used.  It  was  shown  that 
reduction  of  hold-time  to  4  ms  was  not  statistically 
different  from  a  CRT  display.  However,  it  remained 


to  be  determined  what  effect  moving  image  blur  had 
on  the  performance  of  tasks  relevant  to  flight 
simulation  and  what  level  of  blur  reduction  may  be 
acceptable  for  the  performance  of  these  tasks. 

In  the  present  study,  we  have  attempted  to  validate 
our  previous  techniques  by  correlating  them  directly 
with  performance  on  a  simplified  air-to-air  task  that 
may  be  performed  in  high-fidelity  Air  Force  flight 
simulators. 

DISPLAY  CHARACTERIZATION 

A  Christie  Matrix  S+2K  DLP  projector  and  Barco 
909  CRT  projector  were  used  for  testing  and 
experimentation.  The  Christie  DLP  projector  was 
equipped  with  a  function  called  Accuframe,  which 
allowed  temporal-resolution  (i.e.  hold-time)  to  be 
adjusted. 

Methods 

Spatial  Resolution 

Display  spatial  resolution  was  assessed  using 
techniques  adapted  from  accepted  measurement 
standards  (Geri,  Winterbottom  &  Pierce,  2004; 
VESA,  2001).  This  involved  measurement  of 
contrast  for  grille  lines  3,  2,  and  1  pixels  wide. 
Resolution  was  determined  by  dividing  the  number 
of  addressed  lines  by  the  grille  line  width  required  to 
reach  25%  contrast.  Luminance  and  gamma 
measurements  were  also  obtained. 

Temporal  Resolution:  Display  Temporal  Response 
Temporal  resolution  was  measured  using  a 
photodiode  and  Fluke  Scopemeter.  Each  display’s 
response  to  a  30  Hz  test  pattern  was  recorded. 

Temporal  Resolution:  Perceived  Blur 
Observers  were  instructed  to  track  a  test  pattern 
consisting  of  two  white  vertical  lines  (DAC  =  255) 
moving  at  speeds  ranging  from  4  to  45  deg/sec  (100 
to  1200  pixels/sec).  The  background  was  black 
(DAC  =  0),  and  the  lines  were  100  pixels  in 
length,and  1  pixel  wide.  Observers  adjusted  the 
separation  between  the  lines  such  that  a  gap  between 
them  was  just  visible  and  such  that  the  lines  were 
equal  in  width.  If  the  lines  appeared  blurred  (i.e. 
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appeared  to  be  wider  than  1  pixel),  gap  width 
necessarily  increased,  thus  the  adjusted  gap  width 
was  taken  as  a  measure  of  perceived  blur. 

Results 

Michelson  contrast  as  a  function  of  grille  line  width 
for  each  projector  and  two  Accuframe  settings  is 
shown  in  Figure  1.  The  number  of  addressed  pixels 
was  1400  x  1050  and  1280  x  1024  for  the  DLP  and 
CRT,  respectively.  As  shown  in  Figure  1,  contrast 
for  either  display  does  not  drop  below  the  VESA 
criterion  (25%)  for  any  grille  width,  thus  the 
measured  and  nominal  resolutions  are  equal.  Use  of 
the  Accuframe  function  had  little  effect  on  spatial 
resolution. 


Figure  1.  Michelson  contrast  as  a  function  of 
grille  line  width  for  DLP  projector  at  two 
Accuframe  settings  and  for  the  CRT  display. 

Luminance  measurements  for  the  DLP  at  each 
Accframe  setting  and  for  the  CRT  display  were 
obtained.  As  might  be  expected,  increasing  the 
Accuframe  setting  (reducing  hold-time)  also  reduced 
luminance.  Maximum  luminance  for  the  DLP  with 
Accuframe  =  0  was  approximately  100  fL. 

Maximum  DLP  luminance  with  Accuframe  =  254 
was  approximately  34  fL.  CRT  maximum  luminance 
was  approximately  15  fL. 

Shown  in  Figure  2  are  measurements  of  temporal 
response  for  the  DLP  display  at  two  Accuframe 
settings  (AF  =  0,  AF  =254).  Measured  hold-time 
(50%  amplitude)  for  the  CRT  and  the  DLP  using  the 
0,  128,  and  254  Accuframe  settings  was  1.4,  16.6, 
7.4,  and  5.8  milliseconds,  respectively.  Use  of  the 
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Accuframe  function  at  maximum  (AF  =  254)  reduced 
hold-time  by  65%. 
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Figure  2.  Response  of  the  DLP  for  two 
Accuframe  settings  to  a  30  Hz  test  pattern. 

Shown  in  Figure  3  is  perceived  blur  as  line  speed  was 
varied  for  each  display  and  Accuframe  setting.  As 
shown,  decreasing  hold-time  significantly  reduces 
perceived  blur.  Analysis  of  Variance  indicated  that 
the  effects  of  Speed  [F(4,40)=451,  p  <0.001],  Hold¬ 
time  [F(3,40)=395,  p  <  0.001]  and  Speed  x  Hold¬ 
time  [F(12,40)=73,  p  <  0.001]  were  significant.  Post- 
hoc  tests  indicated  that  the  16.6,  and  8.7  msec  Hold- 
times  (AF=0,  and  AF  =  128)  resulted  in  perceived 
blur  that  was  significantly  greater  than  the  CRT  (1.4 
msec  Hold-time).  However,  perceived  blur  for  the 
7.4  msec  Hold-time  (AF  =  254)  was  not  statistically 
different  (averaged  across  speed)  from  that  of  the 
CRT. 
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Results 

Shown  in  Figure  4  is  roll  detection  accuracy  for  each 
roll  angle,  speed,  and  hold-time.  As  shown, 
performance  is  worst  for  small  roll  angles. 
Performance  increased  as  roll  angle  increased. 
Performance  was  also  worst  for  the  highest  speed 
condition.  Figure  5  shows  estimated  threshold  roll 
angles  for  each  target  speed  and  display  Flold-time. 
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Figure  3.  Perceived  blur  as  a  function  of  line 
speed  and  display  Hold-time. 

AIRCRAFT  ROLL  DETECTION 

Methods 

Observers 

Four  observers  with  normal  or  corrected  to  normal 
vision  participated. 


Apparatus  and  stimuli 

The  Barco  CRT  projector  and  Christie  DLP  were 
used  to  project  images  52”  x  43”  (72°  x  62°)  and  52” 
x  29”  (72°  x  44°);  respectively.  Brightness  and 
contrast  were  adjusted  such  that  the  CRT  and  the 
DLP  at  each  Accuframe  setting  were  of 
approximately  equal  luminance.  The  stimulus  was  an 
F-16  model  which  moved  across  the  screen  at  speeds 
of  8.0,  17.5  and  35.0  degrees/sec  (435,  957,  1961 
knots).  The  F-16  always  moved  from  left  to  right. 

Procedure 

The  moving  F-16  model  was  displayed  for 
approximately  2  seconds.  The  F-16  rolled  4,  7,  10, 
13,  16,  or  19  degrees.  The  observers’  task  was  to 
track  the  motion  of  the  aircraft  and  indicate,  using  a 
2-button  response  box,  whether  or  not  the  F-16 
rolled.  Roll  detection  accuracy  was  recorded  for  3 
levels  of  speed,  6  levels  of  roll,  and  4  levels  of  hold¬ 
time.  F-16  simulated  distance  was  always  1600  m  (a 
target  size  of  approximately  34  arcmin  horizontal 
visual  angle).  Observers  viewed  the  display  from  a 
distance  of  36”.  Image  size  and  viewing  distance 
were  selected  to  match  as  closely  as  possible  the 
conditions  in  the  Mobile  Modular  Display  for 
Advanced  Research  and  Training  (M2DART,  see 
e.g.  Wight,  Best,  and  Peppier,  1998). 


Figure  4.  Proportion  correct  roll  detection  for 
each  target  roll  magnitude,  display  Hold-time, 
and  target  speed.  Vertical  lines  indicate  threshold 
estimates  for  roll  detection. 


Figure  5.  Roll  detection  thresholds  for  each  target 
speed  and  display  Hold-time. 

A  repeated  measures  ANOVA  (Huynh-Feldt 
corrected)  indicated  that  the  effect  of  Hold-time  was 
significant  [F(3,  9)  =  6.7,  p  <  0.05],  effect  of  Speed 
was  highly  significant  [F(2,  6)  =  47.4,  p  <  0.001]. 
The  Hold-time  x  Speed  interaction  was  not 
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significant  [F(6,  18)  =  2.4,  p  =  0.13].  Threshold 
generally  increased  with  speed  for  all  Hold-times.  A 
set  of  planned  contrasts  indicated  that  roll  thresholds 
for  the  16.6  ms  Hold-time  were  significantly 
different  [F(l,  3)  =  70.1,  p  <  0.01]  than  those  for  the 
CRT  (1.4  ms  Hold-time),  and  roll  thresholds  for  the 
7.4  ms  Hold-time  were  marginally  different  [F(l,  3) 
=  9.4,  p  =  0.055],  but  that  roll  thresholds  for  the  5.8 
ms  Hold-time  were  not  significantly  different  from 
the  CRT  [F(l,  3)  =  2.0,  p  =  0.25], 

Discussion 

The  results  of  the  roll  detection  task  indicated  that 
display  temporal  resolution  affected  performance  on 
a  representative  air-to-air  task  relevant  to  Air  Force 
flight  simulation.  For  low  temporal  resolution  (i.e. 
increased  Hold-time),  significantly  greater  target  roll 
angle  was  required  for  observer  detection.  Increasing 
temporal  resolution  of  the  DLP  projector  through  the 
use  of  the  AccuFrame  function  significantly 
improved  roll  detection.  For  the  highest  target 
speeds,  performance  was  nearly  equal  for  the  DLP 
and  the  CRT  displays.  Across  all  speeds,  there  was 
no  statistical  difference  in  performance  for  the  5.8  ms 
DLP  Hold-time  and  the  1.4  ms  CRT  Hold-time. 
However,  as  shown  in  Figure  5,  it  appears  that 
performance  with  the  CRT  at  the  lowest  target  speed 
was  better  relative  to  the  DLP. 

Clearly,  however,  performance  on  this  air-to-air  task 
using  a  DLP  with  reduced  Hold-time,  and  at  speeds 
likely  to  be  encountered  in  typical  air-to-air 
scenarios,  was  not  drastically  different  than  that  for 
the  CRT,  which  is  the  display  technology  currently 
used  in  many  Air  Force  flight  simulator  displays.  A 
DLP  display  with  Hold-time  reduced  to 
approximately  5.8  ms  appears  to  allow  acceptable 
performance  for  high  speed  air-to-air  tasks  such  as 
the  one  used  in  this  experiment.  However,  a  more 
rigorous  test  of  digital  display  acceptance  for  high 
speed  flight  simulation  will  be  ground  target 
detection/identification  in  a  low  altitude  flight  task. 
In  this  case,  terrain  blur  will  be  very  noticeable  and 
may  obscure  ground  targets. 

Shown  in  Figure  6  is  display  Hold-time  measured 
using  the  procedure  described  above  versus  Average 
Perceived  Blur  obtained  using  the  Moving  Line  test 
pattern.  These  data  were  obtained  for  a  Barco  909 
CRT,  Christie  Matrix  S+2k  across  three  Accuframe 
settings,  and  for  a  shuttered  LCoS  projector  (see 
Winterbottom,  et  al,  2007).  As  shown,  display  Hold¬ 
time  accurately  predicts  perceived  blur  (r  =  0.963,  p 
<0.001). 


Figure  7  shows  Perceived  Blur  obtained  using  the 
Moving  Line  test  pattern  vs.  F-16  Roll  Threshold 
from  Experiment  1.  As  shown,  measurement  of 
perceived  blur  correlates  very  well  with  the  roll  angle 
required  for  accurate  detection  (r  =  0.82,  p  <  0.001). 


Hold-time  (ms) 


Figure  6.  Average  perceived  blur  as  a  function  of 
Hold-time  for  a  CRT,  DLP  with  Accuframe,  and 
shuttered  LCoS  projector. 

The  correlation  among  these  variables  suggests  that  a 
simple  Hold-time  measurement  can  be  used  to  not 
only  predict  perceived  blur,  but  also  performance  on 
critical  flight  simulation  tasks.  This  allows  for 
minimum  display  temporal  resolution  standards  to 
easily  be  established  and  verified  for  flight 
simulation  applications. 


Perceived  Blur  (arcmin) 


Figure  7.  Perceived  blur  versus  roll  angle 
threshold. 

The  temporal  resolution  measurement  procedure  we 
have  described  here  and  in  previous  work 
(Winterbottom,  et  al,  2004,  2006,  2007)  is  simpler  to 
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administer  and  interpret  than  other  methods,  such  as 
the  Motion  Picture  Response  Time  (e.g.  Igarashi,  et 
al,  2003;  Oka,  &  Enami,  2004;  Igarashi,  et  al,  2004). 
This  procedure  is  also  much  less  expensive,  requiring 
only  a  photodiode  and  Oscilloscope.  Based  on  the 
results  presented  here  and  in  our  previous  work  with 
a  shuttered  LCoS  projector,  digital  displays  with  a 
temporal  resolution  of  5.8  ms  or  better  may  be 
acceptable  for  flight  simulation  applications,  and 
even  displays  with  a  temporal  resolution  of  6  to  8  ms 
may  be  adequate. 

CONCLUSION 

Moving  image  blur  has  typically  only  been  assessed 
using  subjective  ratings,  or  by  measuring,  in  various 
ways,  amount  of  blur  as  a  function  of  speed  and 
display  characteristics.  We  have  described  here,  how 
display  temporal  resolution,  and  hence  moving  image 
blur,  affects  performance  on  a  task  relevant  to  flight 
simulation  and  training.  The  results  presented  here 
indicate  that  sample  and  hold  type  displays  whose 
hold-times  are  reduced  to  5.8  ms  would  likely  be 
acceptable  for  fast-jet  flight  simulation,  and  that  even 
a  hold-time  as  high  as  6  to  8  msec  may  be  adequate. 
Further,  we  show  that  an  easily  obtainable 
measurement,  using  inexpensive  equipment,  is 
sufficient  not  only  to  predict  perceived  blur,  but  also 
performance  on  flight  tasks  affected  by  moving 
image  blur. 

ACKNOWLEDGMENTS 

This  work  was  supported  by  U.S.  Air  Force  contract 
FA8650-05-D-6502.  In  addition,  the  following 
agencies  provided  financial  and/or  technical  support 
for  this  research:  Air  Combat  Command 

(ACC/A8AT),  Secretary  of  the  Air  Force 
International  Affairs  (SAF/IA),  and  Defence 
Research  and  Development  Canada  -  Toronto.  We 
thank  Bill  Morgan  (The  Boeing  Co.)  for  developing 
the  simulation  and  data  collection  software  and  the 
pilots  that  participated  in  each  experiment. 

REFERENCES 

Geri,  G.,  Winterbottom,  M.,  Pierce,  B.  (2004). 
Evaluating  the  spatial  resolution  of  flight 
simulator  visual  displays.  Air  Force  Research 
Laboratory  Technical  Report  Number:  AFRL- 
HE-AZ-TR-2004-0078. 

Geri,  G.  &  Morgan,  B.  (2007).  The  effect  of 
FLCoS-display  hold  time  on  the  perceived  blur  of 
moving  imagery.  Journal  of  the  SID,  15  (1),  pp. 
87-91. 


Igarashi,  Y.,  Yamamoto,  T.,  Tanaka,  Y.,  Someya,  J., 
Nakakura,  Y.,  Yamakawa,  M.,  Hasegawa,  S., 
Nishida,  Y.,  &  Kurita,  T.  (2003).  Proposal  of  the 
perceptive  parameter  motion  picture  response 
time  (MPRT).  SID  03  Digest,  3  1 .2. 

Igarashi,  Y.,  Yamamoto,  T.,  Tanaka,  Y.,  Someya,  J., 
Nakakura,  Y.,  Yamakawa,  M.,  Nishida,  Y.,  & 
Kurita,  T.  (2004).  Summary  of  moving  picture 
response  time  (MPRT)  and  futures.  SID  04 
Digest,  43.3. 

Klompenhouwer,  M.A.  &  Velthoven  (2004).  LCD 
motion  blur  reduction  with  motion  compensated 
inverse  filtering.  SID  04  Digest,  3502,  1340- 
1343. 

Kurita,  T.  (2001).  Moving  picture  quality 
improvement  for  hold-type  AM-LCDs.  SID  04 
Digest,  320,  986-989. 

Nakamura,  H.  &  Sekiya,  K.  (2001).  Overdrive 
method  for  reducing  response  times  of  liquid 
crystal  displays.  SID  01  Digest,  3201,  1256-1259. 

Oka,  K.,  &  Enami,  Y.  (2004).  Moving  picture 
response  time  (MPRT)  measurement  system.  SID 
04  Digest,  43.4. 

Teunissen,  K.,  Li,  X.,  &  Heynderickx,  I.  (2007). 
Measuring  motion  blur  in  liquid  crystal  displays. 
Information  Display,  23  (1),  pp.  20-23. 

VESA  Flat  Panel  Display  Measurements  Standard, 
Version  2,  Video  Electronics  Standards 
Association,  Milpitas,  CA,  June  2001,  pp. 76-77. 

Wight,  D.  R„  Best,  L.  G.,  &  Peppier,  P.  W.  (1998). 
M2DART:  A  Real-Image  Simulator  Visual 
Display  System.  Air  Force  Research  Laboratory’ 
Technical  Report  Number:  AFRL-HE-AZ-TR- 
1998-0097. 

Winterbottom,  M.W.,  Geri,  G.A.,  Morgan,  W.D.  & 
Pierce,  B.J.  (2004).  An  integrated  procedure  for 
measuring  the  spatial  and  temporal  resolution  of 
visual  displays.  I/ITSEC  Conference 

Proceedings,  Paper  No.  1855. 

Winterbottom,  M.,  Geri,  G.,  Morgan,  B.,  Eidman,  C., 
Gaska,  J.,  &  Pierce,  B.  (2006).  Perceptual  tests 
of  the  temporal  properties  of  a  shuttered  LCD 
Projector.  SID  06  Digest,  pp.  494-497. 

Winterbottom,  M.,  Geri,  G.,  Eidman,  C.,  &  Pierce,  B. 
(2007).  Perceptual  Tests  of  the  Temporal 
Response  of  a  Shuttered  LCoS  Projector.  SID  07 
Digest,  pp.  334-337. 


2007  Paper  No.  7274  Page  6  of  7 


Interservice/Industry  Training,  Simulation,  and  Education  Conference  (I/ITSEC)  2007 

Yamamoto,  T.,  Aono,  Y.  &  Tsumura,  M.  (2000). 

Guiding  principles  for  high  quality  motion  picture 
in  AM-LCDs  applicable  to  TV  monitors.  SID  00 
Digest,  3101,  456-459. 


2007  Paper  No.  7274  Page  7  of  7 


